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Measurement  Space  Drill  Support 

Chapter  1 
Introduction 


Overview 

This  document  provides  the  plan,  execution  and  recommendations  for  TRAC-MTRY’s  par¬ 
ticipation  in  as  many  TRAC  Measurement  Space  (MS)  Drill  events  as  possible  from  August 
of  2014  through  February,  2015.  TRAC-MTRY  developed  a  plan  for  support,  receiving  spe¬ 
cific  guidance  from  the  TRAC  Board  of  Directors  (BOD),  executed  the  MS  Drill  support, 
observed  trends  and  made  recommendations  for  future  events. 


Background 

During  the  May  2014  BOD  meeting  in  Monterey,  CA,  TRAC-MTRY  was  charged  with 
figuring  out  how  to  integrate  past  and  current  TRAC-MTRY  research  into  current  and 
future  TRAC  studies.  The  BOD  recommended  the  best  forum  would  be  the  MS  Drill 
events  held  near  the  beginning  of  a  study  time  line.  The  MS  Drill  was  identified  because 
during  these  events  the  tools,  methods,  and  techniques  for  evaluating  the  differentiating 
attributes  are  discussed  between  the  stakeholders,  study  team,  and  other  key  personnel.  For 
more  information  on  MS  Drills,  see  cobpQ  In  order  to  facilitate  the  research  integration, 
a  TRAC-MTRY  analyst  should  be  present.  These  allows  the  analyst  to  gain  a  greater 
understanding  of  the  study  knowing  the  problems,  possible  solutions  and  conditions  of  the 
study.  The  TRAC-MTRY  analyst  is  required  to  have  a  historical  understanding  of  the 
different  research  that  has  been  completed  or  is  in  the  process  by  not  only  TRAC-MTRY 
but  the  other  TRAC  centers.  The  participating  analysts  should  also  be  the  same  for  the 
duration  of  this  project  in  order  to  identify  common  trends  or  make  recommendations  for 
the  future  execution  of  MS  Drills. 


Problem  Statement 

Integrating  TRAC  research  at  a  pivotal  point  in  the  TRAC  studies/projects  life-cycle  through 
a  TRAC-MTRY  analyst  participation  at  the  measurement  space  drill;  with  an  effort  to 
helping  the  process  and  offering  suggestions  where  research  has  been  made  in  applicable 
areas. 
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Issues  for  Analysis 

Issue  1:  What  is  an  effective  way  to  integrate  research  into  current  studies? 

EEA  1.1:  Participate  as  a  contributing  members/  participants  at  the  measurement 
space  event? 

EEA  1.2:  Act  as  a  conduit  between  current  and  past  research  that  could  potentially 
support  the  study? 

Issue  2:  What  are  current  trends  across  all  TRAC  centers  when  conducting  MS  Drill 
events? 

EEA  2.1:  Participate  in  as  many  different  types  of  study  MS  Drill  with  different  center 
leads? 

Constraints,  Limitations,  and  Assumptions 

•  Constraints 

—  Must  be  complet  NLT  June  2015. 

—  Brief  BOD  on  emerging  results  NLT  April  2015. 

•  Limitations 

—  Only  one  dedicated  analyst  available  for  MS  Drills  during  Fall  2014. 

—  Participate  in  only  funded  study  MS  Drills. 

•  Assumptions 

—  TRAC-MTRY  is  notified  by  other  centers  when  MS  Drills  occur. 


Constraints  limit  the  project  team’s  options  to  conduct  the  research.  Limitations  are  a 
project  team’s  inabilities  to  investigate  issues  within  the  sponsor’s  bounds.  Assumptions  are 
research-specific  statements  that  are  taken  as  true  in  the  absence  of  facts. 


Technical  Approach 

Participate  in  as  many  measurement  space  drills  as  feasible  to  act: 

1.  as  a  contributing  members/  participants  at  the  measurement  space  event. 

2.  as  a  conduit  between  current  and  past  research  that  could  potentially  support  the 
study. 

3.  Document  the  lessons  learned  with  recommendations  to  a  sustainable  effort. 
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4.  Identify  common  trends  across  all  of  the  TRAC  centers. 


Steps  one  and  two  are  covered  in  detail  in  Chapter  [2j  while  step  three  is  addressed  in  Chapter 
[3|  Step  4  is  explained  in  Chapter  [4] which  concludes  this  report. 


Timeline 


Aug  2014 
Feb  2015 
Apr  2015 


Begin  measurement  space  support. 
Mid-project  review. 

Final  out-brief  and  write  report. 
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Chapter  2 

MS  Drill  Events  and  Participation 


In  this  chapter,  we  will  cover  the  logistical  execution  of  the  MS  Drill  in  the  first  section  (|2]) 
followed  by  a  discussion  of  the  specific  studies  supported  during  this  effort  ([2j 


Logistics  of  Execution 

Notification  by  Center 

There  were  three  primary  ways  of  notification  about  an  upcoming  MS  Drill: 


•  Through  Center  established  Point  of  Contact  (POC). 

•  Directly  contacted  TEDS  listed  study  POC. 

•  Word  of  mouth. 


The  center  established  POC  for  the  MS  Drill  was  the  official  notification  received  once  an 
event  was  being  planned  at  a  center  and  their  logistics  were  being  coordinated.  This  method 
was  more  effective  at  centers  with  a  robust  operations  section  in  which  center  calendars  and 
drill  support  was  coordinated. 

The  lead  TRAC-MTRY  analyst  contacted  the  listed  study  POC  when  a  project  code  was 
established  within  TEDS.  Often  times  this  would  be  the  study  deputy/XO.  Project  codes 
are  usually  established  even  before  a  study  is  confirmed  as  a  planned  project  so  often  times 
it  would  be  identified  that  an  MS  drill  is  likely  to  occur  but  when  it  would  occur  depended 
on  final  study  approval.  This  process  was  good  for  establishing  initial  contact  and  letting 
the  study  team  know  TRAC-MTRY  was  available  as  part  of  the  team  but  did  not  prove 
fruitful  in  identifying  the  MS  Drill  event. 

Once  the  project  was  established  and  TRAC-MTRY  was  present  at  several  events,  study 
teams  passed  on  information  they  knew  within  TRAC  or  other  studies  they  knew  were 
executing  MS  Drills.  This  helped  with  the  receptiveness  amongst  the  study  teams  and  the 
ease  of  understanding  the  purpose  of  TRAC-MTRY  participation.  Through  word  of  mouth, 
the  TRAC-MTRY  analyst  is  contacted  by  a  study  team  even  prior  to  having  a  project  code. 
This  method  helps  to  inculcate  TRAC-MTRY  analyst  participation  as  part  of  the  fabric  of 
a  study  MS  Drill. 
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Funding 


If  travel  was  necessary,  there  were  two  ways  in  which  an  analyst  was  funded.  First  considera¬ 
tion  for  funding  was  funding  by  study  lead  through  Cross-organizational  Line  Of  Accounting 
(LOA)  with  study  funds  (reimbursable).  As  a  secondary  effort,  the  TRAC-MTRY  Director 
committed  TRAC-MTRY  mission  funds  if  there  were  no  study  funding  available  and  it  was 
a  priority  study. 

The  analyst  time  (hours)  supporting  the  different  studies  were  accounted  for  through  TRAC 
Employee  Data  Site  (TEDS)  for  each  individual  project  code.  The  only  time  against  the  MS 
Drill  support  project  code  was  time  spent  specifically  working  on  this  project  such  as  BOD 
briefings  or  the  writing  of  this  report. 

Support  by  Analyst 

A  TRAC-MTRY  analyst  participated  in  MS  Drills  in  various  ways  to  include  in  person 
through  Temporary  Duty  (TDY)  travel,  teleconference  or  emails  pre/post  events.  The  role 
of  the  analyst  varied  from  recorder  to  active  participant  as  part  of  the  analyst  team  to 
Subject  Matter  Expert  (SME)  conduit.  The  participation  level  at  each  drill  varied.  The 
contributing  factors  to  the  role  of  the  MTRY  analyst  present  were  the  number  of  TRAC 
participants,  the  size  of  the  study,  the  size  of  the  MS  Drill  event,  and  the  facilitator  leading 
the  drill.  Not  all  MS  Drill  support  was  identical  nor  is  it  plausible  that  it  could  be  identical 
in  the  future. 


MS  Drill  Participation 

In  this  section,  we  detail  the  different  types  of  studies/projects  in  which  TRAC-MTRY 
participated  by  center.  In  Figure  [l]  an  aggregate  list  of  drills  participated  in  organized  by 
center  and  TEDS  project  type  code. 

The  current  TEDS  project  codes  are  listed  if  Figure  [2]  One  finding  through  examining  cur¬ 
rent  projects  and  recent  studies/projects  within  TRAC  was  the  project  code  system  should 
be  updated  to  reflect  the  type  of  analysis  TRAC  conducts.  See  Appendix  [D]  for  recommen¬ 
dations  and  analysis. 
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TRAC-MTRY  MS  Drill  Participation fife 


1  .Analysis  of 

Alternatives 

2.  Future  Concept 

Analysis 

3.  Requirements 

Analysis 

4.  Experimentation 

5.  Scenario  &  Data 

Develpoment 

6.  Models  and 

Methods  R&D 

7.  Support  to 

COCOM, 

JIM  Organizations 

8.  Training  Studies 

9.  Other  Studies 

0.  Non-Studies 

TRAC- 

LEE 

AMEDD  Grd  Amb 
ARSOF  Grd  Mob 
Prot  Grd  Amb 

TRAC- 

WSMR 

SAAC  Study  (II) 

MUM-T  Study 

APL/CM  (II) 
USARPAC 
Support 

TRAC- 

FLVN 

Lower  Tier  AMD  (1) 
Lower  Tier  AMD  (II) 
H-47  Block  II  (1) 
*H-47  Block  II  (II) 

AVN  FVL  Att  (1) 
*AVN  FVL  Att  (II) 

TRAC- 

MTRY 

F2025B  Logistic 
Flow 

MS  Drill 
Support 

FY15  Research 

Elicitation 

KM  Implement 

*MS  Drill  occurred  but  TRAC-MTRY  did  not  attend 


•  Participated  in  multiple  types  across  all  TRAC  centers. 

Legend 

•  White  space  with  no  short  title  represents  active  projects: 

1 .  MS  Drill  occurred  prior  to  TRAC-MTRY  MS  Drill  project,  or 

2.  MS  Drill  not  conducted,  or 

3.  MTRY  not  aware  of  MS  Drill. 

Center  has  open  project 

<3  projects  at  center  since  2005 

No  current  project  types  at  center 

Data  as  of  2/28/15 

Figure  1.  Aggregate  roll-up  of  MS  Drill  participation 


TEDS  Current  Code  and  Title 

0 

Non-Studies 

1 

Analysis  of  Alternatives 

2 

Future  Concept  Analysis 

3 

Requirements  Analysis 

4 

Experimentation 

5 

Scenario  Development  &  Data  Development 

6 

Models  and  Methods  R&D 

7 

Support  to  Combatant  Commanders,  Joint,  Interagency,  and  Multinational  Organizations 

8 

Training  Studies 

9 

Other  Studies 

Figure  2.  Current  TEDS  Project  Codes 
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Chapter  3 

Observed  MS  Drill  Flow  of  Events 


Observations  from  participating  in  MS  drills  during  a  condensed  period  of  time  from  August 
2014  through  February  2015  are  compiled  in  the  chapter  to  reflect  the  general  flow  of  a  MS 
Drill.  The  event  is  described  in  three  sections  in  this  chapter,  the  pre-event  activities  [3j  the 
event  execution  [3j  and  the  post-event  activities  [3]  In  no  way  is  this  intended  as  a  prescription 
for  an  MS  Drill  but  intended  as  a  general  description  of  what  was  observed  during  this  period 
of  time.  It  is  accepted  that  each  MS  Drill  is  unique  it  its  occurrence  much  like  every  river  is 
different  but  the  flow  of  the  Drills  followed  some  of  the  same  patterns  much  like  the  flow  of 
water  has  certain  properties. 


Pre-event  Activities 

Prior  to  an  MS  Drill  execution,  there  are  many  pre-activities.  Certain  tasks  are  inherent  to 
hosting  any  workshop  and  are  covered  briefly,  though  not  exhaustively,  followed  by  specific 
tasks  associated  with  the  MS  Drill.  Some  tasks  in  which  the  host  of  the  workshop  should 
complete  include: 


•  Coordinate  for  a  location  at  the  host  site  if  the  MS  Drill  is  a  live  event.  The  location 
should  be: 

—  appropriate  for  the  anticipated  audience,  having  a  room  too  large  or  small  both 
can  detract  from  the  participation. 

—  approved  for  the  security  clearance  required  of  the  workshop. 

—  equipped  with  the  level  of  technology  necessary  such  as  projectors,  white  boards, 
etc. 

•  Complete  the  list  of  invited  attendees  to  include  the  stakeholders,  TRAC  analysts, 
sponsors,  and  other  personnel  specific  to  the  study. 

•  Decide  who  will  fund  the  travel,  if  necessary  of  each  invitee.  In  some  cases,  each 
attendee  must  be  considered  independently  for  funding. 

•  Decide  the  length  of  the  workshop. 

•  Set  goals  of  items  that  need  to  be  completed  while  everyone  is  present  and  establish 
the  timeline  that  will  support  those  goals. 
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Again,  the  list  for  workshop  preparation  is  not  exhaustive  but  highlights  some  of  the  logistical 
concerns  of  hosting  a  workshop.  For  further  information  and  more  detailed  lists  see  the 
TRAC-Analyst  Development  Program  (ADP)  on  workshops.  Each  TRAC-center  operations 
directorate  also  maintains  a  list  for  hosting  workshops  in  general. 

Some  specific  tasks  for  MS  Drill  event  preparation  are: 


•  completing  the  agenda  to  ensure  expectations  of  the  workshop  are  articulated. 

•  hosting  pre-workshop  discussions  with  the  analyst  team,  either  in  person  or  telephonic 
if  the  team  is  distributed. 

•  notifying  participants  of  the  schedule  MS  Drill  and  why  they  are  invited  at  least  thirty 
days  prior  to  allow  for  reaction  time. 

•  creating  templates  of  a  problem  statement,  alternatives,  attributes  and  anything  else 
the  workshop  will  address  so  the  participants  can  have  a  warm  start  when  the  discussion 
begins.  This  particular  step  is  helpful  for  the  stakeholders  and  sponsors  who  are  less 
familiar  with  the  process. 

•  assigning  seating  arrangements  which  can  facilitate  discussions.  Knowing  certain  per¬ 
sonalities  or  organizations  help  with  arranging  the  seats  and  assigning  where  the  par¬ 
ticipants  sit  help  with  the  facilitation. 

•  identify  key  roles  to  be  fulfilled  such  as  the  facilitator,  note  taker,  time  keeper,  etc.,  of 
the  workshop  from  the  analyst  team. 


From  the  tasks  listed,  the  two  most  critical  are  the  notification  and  identification  of  key 
roles.  When  the  workshop  does  not  have  the  right  participants  or  an  incomplete  team,  it 
renders  it  useless. 


Event  Execution 

The  majority  of  the  MS  Drill  events  are  generally  the  first  in-person  meeting  of  stakeholders, 
sponsors,  and  analysts  for  this  specific  study.  This  is  the  time  for  the  study  team  to  scope 
the  problem  and  identify  any  issues  early  in  the  study  process.  Having  everyone  present 
allows  for  clearly  defining  the  way  ahead  and  what  roles  each  participate  will  fulfill. 

In  this  section,  the  execution  and  flow  of  the  event  is  discussed.  Again,  this  is  the  general 
flow  of  events  and  not  prescriptive  of  how  and  event  should  be  run.  This  also  considers  the 
case  in  which  the  event  is  a  live  hosted  MS  Drill  with  most  participants  present. 

The  flow  of  workshops  from  beginning  to  end  generally  followed  this  in  the  order  presented 
below. 
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1.  Welcome  brief  with  the  agenda  for  the  workshop,  the  logistics  of  the  workshop,  and 
any  other  administrative  concerns. 

2.  Discussion  of  alternatives  being  considered  presented  by  stakeholders/sponsors  to  the 
participants. 

3.  A  descriptive  presentation  of  what  a  measurement  space  drill  is  to  include  the  terms 
of  reference  and  goals  for  the  week. 

4.  Scenarios  and  enemy  situation  in  which  the  alternatives  will  be  compared  usually  pre¬ 
sented  by  TRISA  and/or  the  appropriate  scenario  development  division  within  TRAC. 

5.  The  problem  statement  is  either  composed  or  restated. 

6.  The  study  issues  and  essential  elements  are  created  or  re-examined. 

7.  Discussion  of  which  attributes  the  study  should  consider  followed  by  identifying  which 
attributes  are  differentiable  between  the  alternatives  and  how  to  measure  the  attribute. 

8.  When  time  allowed,  the  tools,  methods,  and  techniques  (TMT)  to  model  the  analysis 
for  attribute  measures  are  identified  and  data  requirements  assigned. 


The  majority  of  the  workshops  usually  ended  by  step  seven  or  cut  other  steps  short  in  order 
to  begin  a  cursory  conversation  about  step  eight  due  to  time  constraints.  The  size  of  the 
workshop  also  influenced  the  amount  of  time  needed  for  each  step. 


Post-event  Activities 

Following  the  MS  Drill  workshops,  the  majority  of  the  analyst  team  discussed  the  appropriate 
TMT  and  attributes  identified.  This  is  when  the  team  starts  to  begin  the  analytic  approach 
in  collecting  data  and  providing  structure  to  the  study.  This  was  usually  done  in  person 
at  the  host  center  and  informally  by  the  team.  The  study  lead,  or  designated  analyst,  also 
follows  up  with  anyone  who  had  a  task  assigned  to  them  during  the  workshop. 

The  period  immediately  following  the  MS  Drill  workshop  is  the  springboard  inn  which  the 
trajectory  of  the  study  is  decided.  It  is  important  to  include  all  analyst  that  were  present 
for  the  MS  Drill  in  the  discussions.  Along  with  the  discussion  on  approaches,  the  data 
requirements  are  modified  and  passed  to  the  appropriate  agency. 
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Chapter  4 

Findings  and  Conclusion 


In  this  chapter  the  findings  of  this  project  are  documented,  |4j  For  simplicity  and  ease  of 
reference,  we  use  the  format  of  issue,  discussion,  and  recommendation.  The  chapter  and 
report  concludes  in  section  |4j 


Findings 

Issue:  Facilitator  Training 

Discussion:  From  workshop  to  workshop,  there  was  a  wide  disparity  among  the  fa¬ 
cilitators  approach  and  ability.  The  variance  of  the  facilitation  was  attributed  to  the 
experience  level  of  the  facilitator.  The  more  experienced  facilitators  drove  discussion 
and  kept  the  group  on  topic  running  an  efficient  workshop.  If  a  facilitator  was  inex¬ 
perienced  the  discussions  often  went  of  topic  and  the  workshop  would  be  side-tracked 
ultimately  coming  short  of  the  goals  in  which  they  set  out  to  complete. 

Recommendation:  Offer  facilitator  training  for  analysts  expected  to  lead  workshops. 
Methods  of  implementing  the  training  are  through  the  ADP  or  an  addition  of  a  short 
course  offered  to  the  community  as  a  whole.  The  short  course  would  allow  for  refresher 
training  or  specifics  for  facilitation  for  mid  to  upper  level  analysts. 


Issue:  No  uniform  notification  method  of  MS  Drill  event 

Discussion:  There  is  a  calendar  within  the  CoBP  SharePoint  portal  but  it  is  not 
updated  or  maintained.  The  center  Ops  are  notified  if  they  are  hosting  the  event  since 
a  facility  has  to  be  coordinated.  The  Travel  section  is  notified  if  analysts  are  traveling 
for  the  event.  All  of  these  are  loosely  coordinated  through  ad  hoc  channels  with  no 
congruent  method.  If  there  were  a  formalized  MS  Drill  event  notification  then  visibility 
of  the  events  would  be  raised,  allowing  for  more  input  from  leadership  within  TRAC. 
It  would  also  allow  the  research  community  to  understand  what  the  current  needs  of 
TRAC  analysts  are. 

Recommendation:  It  is  recommended  that  the  center  operations  office  within  TRAC 
maintain  the  SharePoint  calendar  with  upcoming  MS  drills  and  notify  other  centers 
when  they  are  occurring. 


Issue:  No  formalized  center  MS  Drill  SME 
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Discussion:  The  purpose  of  the  MS  Drill  SME  is  to  maintain  current  practices  within 
TRAC,  coordinate  the  MS  drill  across  that  center  and  help  with  the  training  of  new 
analysts  or  study  leads  on  how  to  conduct  an  MS  drill.  There  is  already  informal 
mentoring  for  conducting  MS  drills  but  establishing  center  SME’s  would  codify  the 
process. 

Recommendation:  Institute  an  additional  duty  of  assigning  an  MS  Drill  SME  at 
each  center. 


Issue:  Only  one  prescribed  format  to  conduct  MS  Drill  in  current  published  CoBP 

Discussion:  Critical  events  which  must  be  completed  as  part  of  the  MS  Drill  process 
should  be  identified  for  studies  that  cannot  host  a  week  long  MS  Drill.  Although  there 
is  danger  in  creating  an  abbreviated  version  because  most  studies  are  short  on  time 
and  would  gravitate  to  this  version,  it  is  necessary.  The  discriminating  attribute  for 
whether  an  abbreviated  or  deliberate  MS  drill  should  be  conducted  is  whether  the 
study  is  funded  or  not.  There  were  multiple  times  a  study  was  not  funded  in  which  a 
MS  drill  was  conducted  through  informal  interviews  or  discussions.  Having  references 
for  these  analysts  within  the  CoBP  would  ensure  the  key  functions  of  an  MS  drill  are 
performed. 

Recommendation:  Develop  an  addition  to  the  MS  Drill  CoBP  which  includes  both 
abbreviated  and  deliberate  MS  drills. 


Conclusion 

During  this  project,  We  participated  in  multiple  styles  of  MS  Drill  events  across  all  centers, 
with  some  being  led  by  a  TRAC  analyst  and  others  led  by  outside  organizations  with  TRAC 
analysts  present.  We  have  captured  the  common  observations  in  hopes  of  improving  the  MS 
Drill  events  in  the  future. 

This  project  is  the  continuation  of  discussions  surrounding  the  conduct  of  MS  Drills  and  is 
in  no  way  the  conclusion  of  this  conversation  among  analyst.  This  is  a  catalyst  for  ways  in 
which  the  process  can  be  improved  to  serve  the  leaders  in  which  the  analysis  is  conducted. 
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Measurement  Space  Drill  Support 


Problem  Statement: 

Integrating  TRAC  research  at  a  pivotal  point  in 
the  TRAC  studies/projects  lifecycle  through  a 
TRAC-MTRY  analyst  participation  at  the 
measurement  space  drill;  with  an  effort  to  helping 
the  process  and  offering  suggestions  where 
research  has  been  made  in  applicable  areas. 

Sponsor:  TRAC-MRO. 

Stakeholders:  TRAC-WSMR,  TRAC-LEE, 
TRAC-FLVN,  TRAC-MTRY. 


Technical  Approach: 

•  Participate  in  as  many  measurement  space 
drills  as  feasible  to  act: 

•  as  a  contributing  members/  participants 
at  the  measurement  space  event. 

•  as  a  conduit  between  current  and  past 
research  that  could  potentially  support 
the  study. 

•  Document  the  lessons  learned  with 
recommendations  to  a  sustainable  effort. 


Measurement  Space  Thinking 


Key  Project  Dates: 


1.  What  is  the  problem? 

2.  What  conditions  in  the  operational  environment  contributed  to  the 
problem  in  the  first  place? 

3.  What  is  it  about  the  solutions  that  promises  to  remedy  the  problem? 

4.  Are  the  attributes  that  differentiate  the  potential  solutions  going  to 
reveal  themselves  under  the  conditions  most  relevant  to  the 
problem? 

5.  Are  those  conditions  prevalent  enough  in  the  operational 
environment  to  justify  revealing  (through  analysis)  the  benefits  that 
discriminate  among  the  solutions? 

Those  conditions  that  Solutions  that  have 

contributed  to  the  Problem  attributes  relevant  to  the 

problem,  but  for  which  the  problem  but  outside  of 

solutions’  attributes  don’t  - >  +— "  those  conditions  that 

pertain,  and  therefore  will  MS  contributed  to  the  problem, 

not  distinguish  among  Conditions  Solutions 

the  solutions.  ♦ 


Conditions  that  discriminate  among  those  solutions’ 
attributes  but  are  not  relevant  to  the  problem. 


12  Aug  2014  Begin  measurement  space 

support. 

15  Jan  2015  Mid-project  review. 

15  Jun  2015  Final  out-brief  and  write  report. 

Deliverables: 

•  Research  integration  into  the  study  process. 

•  Present  study  needs  to  the  research  community. 

•  Document  TRAC’s  measurement  space  drills 
lessons  learned  over  the  course  of  10  months. 

•  Document  applicable  research  into  the  study 
process. 
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TRAC-MTRY  Measurement  Space 

Drill  Support 

Project  Code  060122 


Project  brief  to  the  TRAC-BOD 

14  April  2015 


i  h  i 


Purpose  and  Agenda 


Purpose:  Provide  an  update  of  TRAC-MTRY’s  Measurement  Space  (MS) 
drill  support  from  Aug  14  through  Feb  15  and  provide  recommendations 
for  future  participation. 


Agenda: 

•  Execution  of  MS  Drill  Support 


•  TRAC-MTRY  MS  Drill  Participation. 


•  MS  Drill  Trends  Observed. 


•  Continued  MTRY  MS  Drill  Support. 
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Execution  of  MS  Drill  Support 


i  h  i 


•  Notification  of  MS  Drills. 

-Through  Center  established  POCs. 
-Directly  contacted  TEDS  listed  study  POC. 
-Word  of  mouth. 


•  Funding  (if  TDY  required). 

-  Project  funded  by  study  lead  through  Cross-org  LOA. 

-  Mission  funded  from  TRAC-MTRY  if  no  project  funding. 


•  Support 

-Attended  MS  Drill  events  in  person  (TDY). 
-Teleconferenced  and  emailed  for  pre/post-MS  Drills. 
-Not  all  MS  Drill  support  was  identical 
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TRAC-MTRY  MS  Drill  Participation 


1 .  Analysis  of 
Alternatives 

2.  Future  Concept 
Analysis 

3.  Requirements 
Analysis 

4.  Experimentation 

5.  Scenario  &  Data 

Develpoment 

6.  Models  and 

Methods  R&D 

7.  Support  to 

COCOM, 

JIM  Organizations 

8.  Training  Studies 

9.  Other  Studies 

0.  Non-Studies 

TRAC- 

LEE 

AMEDD  Grd  Amb 
ARSOF  Grd  Mob 
Prot  Grd  Amb 

TRAC- 

WSMR 

SAAC  Study  (II) 

MUM-T  Study 

APL/CM  (II) 
USARPAC 
Support 

TRAC- 

FLVN 

Lower  Tier  AMD  (1) 
Lower  Tier  AMD  (II) 
H-47  Block  II  (1) 
*1-1-47  Block  II  (II) 

AVN  FVLAtt  (1) 
*AVN  FVLAtt  (II) 

TRAC- 

MTRY 

F2025B  Logistic 
Flow 

MS  Drill 
Support 

FY15  Research 
Elicitation 

KM  Implement 

*MS  Drill  occurred  but  TRAC-MTRY  did  not  attend 


•  Participated  in  multiple  types  across  all  TRAC  centers. 

•  White  space  with  no  short  title  represents  active  projects: 

1 .  MS  Drill  occurred  prior  to  TRAC-MTRY  MS  Drill  project,  or 

2.  MS  Drill  not  conducted,  or 

3.  MTRY  not  aware  of  MS  Drill. 


_ Legend _ 

Center  has  open  project 
<3  projects  at  center  since  2005 
No  current  project  types  at  center 
Data  as  of  2/28/15 
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MS  Drill  Trends  Observed 


BLUF:  Each  drill  was  different  in  topic  yet  organized  with  the  framework 
established  by  the  MS  Community  of  Best  Practice  (CoBP). 


•  Flow  of  the  workshops. 

-First  in-person  meeting  of  stakeholders/sponsors/analysts. 

-Stakeholder  shaping  meeting. 

-Did  not  thoroughly  identify  or  discuss  the  tools,  methods,  and  techniques 
(TMT)  for  analysis  during  formal  workshop 
-Analyst  team  identifies  TMT  post-event  (Exception  when  two  drills 
conducted). 

•  Facilitation  Techniques. 

-Varied  by  experience. 

-  Do  not  teleconference,  if  at  all  possible. 

-Collaborated  with  Workshop  lead,  Amy  McGrath,  for  the  Analyst  Development 
Program  (ADP)  for  MS  drill  input. 

•  Initial  Findings 

-Recommend  improved  facilitator  training.  (ADP,  Short  Course,  etc.) 

-MS  Drill  SME  at  each  center. 

-Develop  abbreviated  and  deliberate  MS  drills.  (Addition  to  CoBP) 

-Formalize  MS  Drill  event  notification.  (Through  Ops  channels) 

Note:  Technical  Report  expected  to  be  published  by  May  2015  with  more  detail 
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Continued  MTRY  MS  Drill  Support 


Figure  out  how  to  integrate  past  and  current  TRAC-MTRY  research  into 
current  and  future  TRAC  studies  -  Paraphrased  from  May  14  BOD 


MS  Drill  Support  Project 

1 .  MS  Drills  are  where  the  tools  and  techniques  for  analysis  are 
identified. 

2.  TRAC-MTRY  will  participate  in  as  many  MS  drills  as  feasible  within  a 
defined  timeline. 


Enduring  MS  Drill  Support 

1 .  Recommend  TRAC-MTRY  participate  in  the  formal  MS  Drill  and  the 
analysts  discussion  following  the  drill. 

2.  Recommend  TRAC  studies  be  identified  through  TRAC-MTRY 
engagement  officer  during  monthly  Ops  meeting. 

3.  Recommend  TRAC-MTRY  participate  in  0-2  MS  Drills  per  month 
across  all  centers,  prioritized  through  collaborative  efforts. 
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Discussion  and  Questions 


POC  is  LTC  Mike  Teter 
Michael.d.teter6.mil@mail.mil 
831-656-7580 
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Appendix  C 

Sample  Measurement  Space  Introduction  briefing 
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o 

Study  Name 

Measurement  Space  Introduction 


Audience 
XX  XXX  2014 
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MS  Overview 


Measurement  space  is  the  set  of  operating  conditions,  when 
adequately  accounted  for  in  analysis  methods,  that  will 
most  likely  distinguish  between  two  or  more  alternatives. 


•  The  measurement  space  concept  is  a  way  to  think  about  the  problem  and 
how  best  to  analyze  it;  it  is  not  meant  to  be  rigid,  prescriptive  or 
formulistic. 


•  The  systematic  consideration  of  measurement  space  focuses  the  analyst 
to  “think  first,  before  doing  anything  else.” 

•  Measurement  space  development  offers  a  valuable  way  to  collaborate 
and  reach  shared  understanding  among: 

-Analysts,  Modelers,  Scenario  Developers,  Warfighters,  System 
Engineers/Developers,  and  Acquisition  Managers. 

•  Measurement  space  influences  study  methodology  and  informs 
scenarios,  methods,  models,  and  tools  (MMT),  and  data  requirements. 


24  Sep  2014 


Unclassified 


2 


C-3 


Terms  of  Reference 


•  Tipping  Point  Issue:  The  principal  issue  or  question  faced  by  the  decision  maker  that  when 
answered,  causes  him/her  to  select  a  particular  course-of-action  over  another.  (MS  Code  of 
Best  Practices  (CoBP)) 

•  Attribute:  A  quantitative  or  qualitative  characteristic  of  an  element  or  its  actions.  (CJCSI 
31 70.01  G,  JCIDS,  1  Mar  09).  In  MS,  it  pertains  to  a  course-of-action  or  solution. 

•  Alternative:  A  potential  DOTMLPF  solution  to  the  problem. 

•  Operational  Impact:  A  hypothesis  of  the  expected  benefit(s)  or  detriment(s)  the  attribute 
brings  to  the  commander. 

•  Measure:  A  criterion  used  to  assess  changes  in  system  behavior,  capability,  or  operation 
environment  that  is  tied  to  measuring  the  attainment  of  an  end  state,  achievement  of  an 
objective,  or  creation  of  an  effect.  (JP  3-0) 

•  Method:  A  systematic  procedure,  [analytic]  technique,  or  mode  of  inquiry  employed  y  or 
proper  to  a  particular  discipline  or  art.  (MS  CoBP) 

•  Model:  A  physical,  mathematical,  or  otherwise  logical  representation  of  a  system,  entity, 
phenomenon,  or  process.  (AR  5-11,  1  Feb  05) 

•  Tool:  Something  (as  an  instrument  or  apparatus)  used  in  performing  an  operation  or 
necessary  in  the  practice  of  a  vocation  or  profession.  (MS  CoBP) 

•  Operational  Scenario:  A  graphic  and  narrative  description  of  the  operational  variables, 
political,  military,  economic,  social,  information,  infrastructure  plus  physical  environment  and 
time;  it  concerns  events  of  a  future  hypothetical  operation.  (TRADOC  Reg  71-4) 


DOTMLPF-  Doctrine,  Organizational,  Training,  Materiel,  Leadership  and  Education,  Personnel,  Facility 
24  sep  2014  Unclassified  3 
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Approaching  Measurement  Space 


1 .  What  is  the  problem? 

2.  What  conditions  in  the  environment  contributed  to  the  problem  in  the  first 
place? 

3.  What  is  it  about  the  solutions  that  promises  to  remedy  the  problem? 

4.  Are  the  attributes  that  differentiate  the  potential  solutions  going  to  reveal 
themselves  under  the  conditions  most  relevant  to  the  problem? 


5.  Are  those  conditions  prevalent  enough  in  the  operational  environment  to 
justify  revealing  (through  analysis)  the  benefits  that  discriminate  among  the 
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Problem  Statement 
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•  A  solid  problem  statement  should  be  developed  prior  to  the 
workshop  and  should  be  a  study  team  effort.  It  should  follow  the 
below  guidelines: 

•  Concise.  The  essence  of  your  problem  needs  to  be  condensed  down  to 
one  or  two  sentence(s).  The  problem  statement  should  be  easily 
understood  by  someone  not  intimately  involved  with  the  study. 

•  Specific.  The  problem  statement  should  focus  your  thinking,  research, 
and  solutions  toward  a  single  issue. 

•  Measurable.  Problems  can  be  measured  in  terms  of  degree  and 
frequency.  The  strongest  problem  statements  incorporate  measurable 
aspects  of  both  the  degree  and  frequency  of  the  problem  as  it  exists. 

•  Impact.  The  problem  statement  should  identify  the  population  affected  by 
the  problem. 

Once  you  identify  your  problem,  the  problem  statement  should  be  a  pithy, 
focused  summary  of  the  relevant  details  that  allows  a  decision  maker  to 
quickly  assess  if  the  study  is  addressing  the  relevant  issues  and 
communicate  to  those  outside  the  team  the  intent  of  the  effort. 

24  sep  2014  Unclassified  5 
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Measurement  Space  Determination 


•  Measurement  space  identifies  the  analytic  intersection  of  the  problem, 
the  attributes  of  potential  solution(s),  and  the  operational  conditions  that 
contribute  to  the  problem. 


•  This  intersection  is  the  space  where  the  distinguishing  attributes  of  the 
solution(s)  will  most  likely  reveal  themselves  in  operationally  relevant 
ways  that  can  be  measured  by  the  analysis  methodology. 


Measurement  Space  Determination  Measurement  Space 

Components 


•  Establishing  the  measurement  space  informs  the  methodology 
requirements  for  the  definition  of  the  operational  context  (e.g.,  scenarios/ 
vignettes),  the  functionality/features  of  MMT,  and  the  types  of  data. 


MMT-  Methods,  Models,  Tools 

CLA-  Constraints,  Limitations,  Assumptions 

24  sep  2014  Unclassified  6 
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Incorporating  Workshop  Products  in 

Analysis 


•  The  conclusion  of  a  MS  workshop  does  not  mean  that  MS  is  completed 
for  a  study. 

•  The  results  of  the  workshop  drive  the  study  methodology  as  well  as  the 
potential 

-Scenario(s)  selection. 

-Data  requirements. 

-Methods,  models,  and  tools  utilized. 


•  The  scenario  representatives  will  identify  the  scenarios  that  contain  the 
desired  conditions  (physical  environment,  threat  capabilities  and  blue 
missions). 

•  The  data  representative  determines  what  data  will  be  needed  from  other 
organizations  based  on  the  measures  and  attributes  discussed  in  the 
workshop  and  the  requirements  of  the  anticipated  MMT. 

•  The  MMT  leads  will  determine  if  a  model  is  necessary  to  answer  the 
problem  and  if  so,  what  model  best  meets  the  requirements  based  on  the 
measures  and  attributes  discussed  in  the  workshop.  This  includes 
combat,  sustainment,  risk,  trades  as  well  as  performance  modeling. 


Unclassified 


Measurement  Space  Workshop 


i  h  i 


Summary 


•  MS  is  not  “easy”;  it  requires  an  in-depth  understanding  of  the  subject  and 
collaboration  amongst  a  wide  range  of  disciplines.  Everyone  must 
contribute  and  share  their  knowledge  with  the  group. 

•  Understanding  the  problem,  its  related  conditions,  and  potential  solutions, 
as  well  as  participation  of  the  correct  attendees,  are  critical  to  a 
successful  MS  workshop. 

•  The  results  of  the  workshop  will  influence  the  study  methodology  as  well 
as  the  potential  models  and  tools  utilized,  data  requirements,  and  the 
scenario  selection  (and/or  vignettes  developed). 

•  A  thorough  MS  workshop  improves  the  quality  of  the  analysis  through 
better  up-front  planning  and  more  efficient  resource  utilization. 


Xx  xxx  2014 
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Example 
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MS  Practical  Example 


•  Initial  problem  statement:  I’m  thirsty  on  my  way  to  work. 

•  Revised  problem  statement:  I  would  like  to  carry  and  drink  a  warm 
beverage  on  my  daily  one  hour  drive  to  work  without  spending  a  lot  of 
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MS  Practical  Example 


•  Initial  problem  statement:  I’m  thirsty  on  my  way  to  work. 


•  Revised  problem  statement:  I  started  a  new  job  with  a  longer  commute 
and  I  would  like  to  carry  and  drink  a  warm  beverage  on  my  one  hour 
commute  to  work  everyday. 

•  Possible  alternatives: 


-Coffee  mug. 
-Insulated  coffee  mug. 
-Thermos. 

-Coffee  shop. 


Attributes 

Operational 

Measures 

Conditions 

Impact 

Environment 

Mission 

Threat 

Friendly 

Echelon 

Time 

Capacity, 

Commute 

time, 

temperature, 
aesthetics, 
safety,  etc. 

Unable  to  enjoy 
beverage  for 
entire  commute. 

Causes 

additional  stops 
on  commute. 

Unable  to  fit  in 
cup  holder. 

Ounces 

Size  of  cup 
holder 

Shape  of  cup 

Winter 

Commute  to 

work. 

Co-drinker, 

gloves, 

"shaky" 

hands. 

Car, 

Cupholder 

Individual 

One  hour 

24  Sep  2014 
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MS  Practical  Exercise  (AAS) 


•  Initial  problem  statement:  The  current  reconnaissance  platform  has  significant 
operational  limitations. 

•  Revised  problem  statement:  OH-58D  suffers  from  the  following  obsolescence 
issues:  limited  capacity  to  upgrade,  severe  operating  limitations  in  high,  hot 
environments,  does  not  have  the  speed,  range,  and  endurance  to  fully  support  the 
full  range  of  Army  operations. 

•  Possible  alternatives: 


-Upgrade  current  platform. 

-  Repurpose  existing  airframes. 

-  Pursue  a  new  start  system. 


Operational  Conditions  (METT-TC) 

Attributes 

Impact  Statement 

Measures 

Environment 

Mission 

Threat 

Friendly 

Ground 

AAS 

Echelon 

Time 

Performance 
(Speed,  range, 
endurance, 
power  margin) 

Provides  the  commander  with 
greater  situational  awareness  of  an 

AO  and  reduced  reaction  time  and 

maneuver  space. 

Ensure  that  the  commander  does 
not  have  to  use  other  AVN  /ground 
assets  to  cover  the  area,  freeing 
these  assets  to  perform  other 
missions  and  potentially  enhancing 
the  combat  power/overall 
survivability  for  the  DIV. 

Continuous  and  Total  On  Station  Time 

Teams  required 

Total  team  flights  required 

FARPs  turns/FARPs  required 

Number  of  teams  required 

#  Threat  detected/  undetected 

#  Friendly  (main  body)  detections  by  threat 

#  Blue  losses 

Timeliness  of  ACQ/Report 

Unplanned  engagements  by  threat 

Time  to  complete  MSN 

Actionable  spot  reports 

Calls  for  fire 

Terrain  coverage 

High- 

Mountainous 
(rolling  hills?) 
Hot-Desert 
Urban/Natur 
al  "Clutter" 
Wind 

Snow/Ice 

Security 

(Screen) 

AD  threat 
(low  alt, 
NOE) 

SAF  threat 

(higher 

alt) 

FARPs 

required 

(moving 

with 

ground 

unit) 

Moving 

to  OBJ 

Ground 

screen 

Troop, 

Squadron, 

CAB 

Support 

elements 

24-36  hrs. 
for  a  TRP 
(or 

squadron) 
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Appendix  D 

Recommendation  to  update  the  TEDS  project  types 


Current  project  codes 


After  attending  several  MS  drills,  a  trend  in  types  of  projects/studies  of  TRAC  were  ana¬ 
lyzed  using  data  within  TEDS.  The  active  studies  are  displayed  by  type  in  Figure  D-l  and 
compared  to  the  MS  drill  events  attended  by  TRAC-MTRY. 


"0"  "1"  "2"  "3"  "4"  "5"  "6"  "7"  "8"  "9" 

TYPE  ▼ 


Figure  D— 1.  Total  active  TRAC  studies/projects  in  red  and  the  studies/projects 
a  TRAC-MTRY  analyst  participated  between  the  dates  of  Aug  14  and  Feb  15 
in  blue.  Source:  TEDS 


Since  this  only  accounts  for  current  studies,  we  decided  to  look  at  studies  completed  within 
the  last  five  years  and  found  there  was  a  trend  of  what  types  of  studies  TRAC  is  doing 
now.  Missions  of  organizations  change  over  time  and  the  types  of  projects  should  reflect  the 
mission  of  the  organization.  Figure  D-2  displays  the  type  of  studies  completed  by  center 


over  the  last  five  years.  This  demonstrates  the  lack  of  certain  types  of  projects  completed. 
For  example  there  were  no  project  type  “8”  completed. 


Recommended  changes 


A  recommended  format  for  changing  the  project  codes  is  presented  in  Figure  |D  3  This 


crosswalks  the  current  codes  with  recommended  updated  codes  and  also  displays  the  project 
types  recommended  for  deletion. 


An  example  using  current  projects  on  how  to  assign  them  to  new  types  is  presented  in  Figure 


D-l 


Sum  of  FLVN  SumofWSMR  Sum  of  LEE  Sum  of  MTRY  Sum  of  HQ 
25 


Values 

■  Sum  of  FLVN 

■  SumofWSMR 

■  Sum  of  LEE 

■  Sum  of  MTRY 

■  Sum  of  HQ 


"Q"  "l”  "2"  "3"  "4"  "5"  "6"  "7"  "8"  "9" 


Figure  D— 2.  Total  studies  TRAC  has  done  from  2010  through  2015  binned  by 
center.  Source:  TEDS 


Crosswalk  TEDS  with  recommended 
Project  Code  Type 


New  Title 

Current  Code  and  Title 

JCIDS  Support 

1 

Analysis  of  Alternatives 

3 

Requirements  Analysis 

Policy  Support 

7 

Support  to  Combatant  Commanders,  Joint,  Interagency,  and  Multinational 
Organizations 

9 

Other  Studies 

Futures 

2 

Future  Concept  Analysis 

Scenarios 

5 

Scenario  Development  &  Data  Develpoment 

Research 

6 

Models  and  Methods  R8iD 

Admin 

0 

Non-Studies 

Delete 

4 

Experimentation 

8 

Training  Studies 

Figure  D— 3.  Recommended  new  study /project  types  cross- walked  with  current 
project  types 


D-4  These  are  the  MS  Drill  events  attended  and  categorized  by  the  recommended  change 


to  project  types. 
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Measurement  Space  Drills  TRAC-MTRY 
Participated 


JCIDS  Support 

Policy  Support 

Futures 

Scenario  Dev. 
&  Data  Dev. 

Admin 

Research 

TRAC-LEE 

AMEDD  Ground 
Ambulance 

USASOC  Ground  Mobility 
Protected  Ground 
Ambulance 

2  Recorded 
Projects/Studies 

2  Recorded 
Projects/Studies 

TRAC- 

WSMR 

SAAC  Study  (II) 

APL/CM  (II) 
USARPAC  Support 

MUM-T  Study 

TRAC- 

FLVN 

Lower  Tier  AMD 
Capability  (I) 

Lower  Tier  AMD 
Capability  (II) 

H-47  Block  11(1) 
*H-47  Block  11(11) 

AVN  FVL  Attributes  (1) 
*AVN  FVL  Attributes  (II) 

TRAC- 

MTRY 

No  Recorded 
Projects/Studies 

F2025B  Logistic  Flow 

2  Recorded 
Projects/Studies 

FY15  Research  Elicitation 
AKM  Implementation 

MS  Drill  Support 
CXXI-EASE  Integration 

*Occurred  but  did  not  attend. 
APlanned  for  future  execution. 


>3  ever  recorded  project  types  at  center 

No  current  project  types  at  center _ 


Figure  D  4.  New  Classification  of  MS  Drill  events  participated  in  by  TRAC- 
MTRY  analyst 
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Appendix  F 
Glossary 


ADP 

Analyst  Development  Program 

BOD 

Board  of  Directors 

CoBP 

Community  of  Best  Practice 

EEA 

Essential  Elements  of  Analysis 

FLVN 

Fort  Leavenworth 

FY 

Fiscal  Year 

LEE 

Fort  Lee 

LOA 

Line  of  Accounting 

MRO 

Methods  and  Research  Office 

MS 

Measurement  Space 

MTRY 

Monterey 

POC 

Point  of  Contact 

SME 

Subject  Matter  Expert 

TDY 

Temporary  Duty 

TEDS 

TRAC  Employee  Database  Site 

TMT 

Tools,  Methods,  and  Techniques 

TRAC 

Training  and  Doctrine  Command  Analysis  Center 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

F-l 


